The elimination of nuisance parameters

We review the Bayesian approach to the problem of the elimination of nuisance parameters from a statistical model. Many Bayesian statisticians feel that the framework of Bayesian statistics is so clear and simple that the elimination of nuisance parameters should not be considered a problem: one has simply to compute the marginal posterior distribution of the parameter of interest. However we will argue that this exercise need not be so simple from a practical perspective. The paper is divided in two main parts: the first deals with regular parametric models whereas the second will focus on non regular problem, including the so-called Neyman and Scott’s class of models and semiparametric models where the nuisance parameter lies in an infinite dimensional space. Finally we relate the issues of the elimination of nuisance parameters to other, apparently different, problems. Occasionally, we will mention non Bayesian treatment of nuisance parameters, mainly for comparative analyses.